Papers with visual reasoning

44 papers
LaMI: Augmenting Large Language Models via Late Multi-Image Fusion (2026.acl-short)

Copied to clipboard

Challenge: Large Language Models lack visual grounding on visual reasoning, despite training on text alone.
Approach: They propose a late multi-image fusion method that augments LLMs with test-time visual signals.
Outcome: Using a late multi-image fusion method, the proposed model outperforms LLMs on visual reasoning and matches VLMs in vision-based tasks.
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models (2024.naacl-long)

Copied to clipboard

Challenge: Vision-language models have demonstrated strong efficacy as visual assistants . however, evaluation of their reasoning capabilities requires a costly benchmark .
Approach: They propose a pipeline to measure the reasoning consistency of vision-language models . they propose supervised fine-tuning of VLMs and feedback from LLMs .
Outcome: The proposed framework reduces cost while ensuring the generation of a high-quality dataset.
VisDoT : Enhancing Visual Reasoning through Human-Like Interpretation Grounding and Decomposition of Thought (2026.findings-eacl)

Copied to clipboard

Challenge: Lack of perceptual grounding limits vision-language models' ability to interpret visual data . prior work on visualized data understanding focused on adapting VLMs to instruction tuning and chain-of-thought supervision .
Approach: They propose a framework that enhances visual reasoning through human-like interpretation grounding.
Outcome: The proposed framework improves on ChartQA and ChartQAPro benchmarks by +11.2%.
Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages (2023.acl-short)

Copied to clipboard

Challenge: Existing studies have shown that the pre-training in English does not transfer well to other languages in a zero-shot setting.
Approach: They propose a simple yet efficient approach to adapt VLP to unseen languages using MPLM.
Outcome: The proposed approach outperforms state-of-the-art models without large parallel corpora across three tasks.
Contextual Modulation for Relation-Level Metaphor Identification (2020.findings-emnlp)

Copied to clipboard

Challenge: Existing approaches to identifying metaphors in text ignore context where metaphor occurs . existing approaches focus on word-level identification without explicitly modelling interaction between metaphor components .
Approach: They propose a method for identifying relation-level metaphoric expressions of certain grammatical relations based on contextual modulation.
Outcome: The proposed architecture achieves state-of-the-art results on benchmark datasets.
FastV-RAG: Towards Fast and Fine-Grained Video QA with Retrieval-Augmented Generation (2026.acl-long)

Copied to clipboard

Challenge: Existing methods for retrieval-augmented generation are inefficient and often fail to maintain high answer quality.
Approach: They propose an efficient VLM-based RAG framework built on a speculative decoding pipeline and a similarity-based filtering strategy to mitigate errors.
Outcome: The proposed framework reduces inference latency without sacrificing correctness . it achieves comparable or higher accuracy than standard approaches while speeding up inference by approximately 2x .
MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems (2024.findings-emnlp)

Copied to clipboard

Challenge: Programming often involves translating detailed and complex specifications into code . current state-of-the-art models struggle to solve these problems, a new study shows .
Approach: They propose a multi-modal coding dataset to evaluate algorithmic problem-solving skills in visually rich contexts.
Outcome: The proposed model lacks powerful vision-code models due to the extreme demand for reasoning abilities.
Detect, Disambiguate, and Translate: On-Demand Visual Reasoning for Multimodal Machine Translation with Large Vision-Language Models (2025.naacl-long)

Copied to clipboard

Challenge: Multimodal machine translation (MMT) aims to leverage additional modalities beyond text . current MMT systems rely heavily on monolingual English captioning data .
Approach: They propose a reasoning-based framework to leverage large-scale vision-language models for MMT . they propose Detect, Disambiguate, and Translate framework to detect ambiguity in input sentence .
Outcome: The proposed framework outperforms state-of-the-art models in disambiguation accuracy and translation quality.
LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression (2025.findings-naacl)

Copied to clipboard

Challenge: LVLMs have been shown to perform well on simple uni-modal benchmarks, but their detailed study on multi-modal models is still lacking.
Approach: They propose a framework to analyze the impact of compression on LVLMs on multi-modal input driven tasks.
Outcome: The proposed framework analyzes the impact of compression on generative performance of large vision language models on multi-modal input driven tasks.
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark (2025.findings-naacl)

Copied to clipboard

Challenge: Recent years have witnessed a significant interest in developing large multimodal models capable of performing various visual reasoning and understanding tasks.
Approach: They propose to use Arabic as a language to evaluate large multi-modal models capable of performing visual reasoning and understanding tasks.
Outcome: The proposed benchmark comprises eight diverse domains and 38 sub-domains to represent a large population of over 400 million speakers.
Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)

Copied to clipboard

Challenge: Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability.
Approach: They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning.
Outcome: The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models.
MM-ChatAlign: A Novel Multimodal Reasoning Framework based on Large Language Models for Entity Alignment (2024.findings-emnlp)

Copied to clipboard

Challenge: Existing MMEA methods rely on knowledge representation learning (KRL) to measure the similarity of entity embeddings.
Approach: They propose a framework that utilizes the visual reasoning abilities of MLLMs for multimodal entity alignment.
Outcome: The proposed framework integrates the visual reasoning abilities of MLLMs for multimodal entity alignment.
Distill Visual Chart Reasoning Ability from LLMs to MLLMs (2025.findings-emnlp)

Copied to clipboard

Challenge: a new method for generating chart annotations is proposed to improve visual reasoning in multimodal large language models.
Approach: They propose a code-as-intermediary translation method for distilling visual reasoning abilities from LLMs to MLLMs.
Outcome: The proposed method is cost-effective, efficient and scalable.
VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism (2025.acl-long)

Copied to clipboard

Challenge: Large Vision-Language Models (LVLMs) have shown exceptional performance in multimodal tasks, but their effectiveness in complex visual reasoning is constrained.
Approach: They propose a training-free approach that enhances Reasoning in Large Vision-Language Models . they propose integrating Monte Carlo Tree Search and Self-Reward mechanisms into the reasoning tree .
Outcome: The proposed approach surpasses current prompting methods and secures state-of-the-art performance across three multimodal reasoning benchmarks.
Simple-VGC: Enhancing Visual Grounding in Multimodal Reasoning via Adaptive Tool Composition (2026.acl-long)

Copied to clipboard

Challenge: Existing multimodal large language models suffer from systematic failures in basic visual understanding.
Approach: They propose a tool-augmented reasoning framework with three targeted compensation strategies to address these problems.
Outcome: The proposed framework improves visual grounding by re-injecting the original image to mitigate visual forgetting, the authors show . the proposed framework also improves the accuracy of the visual inputs, the researchers show - and the results are promising .
Think Visually: Question Answering through Virtual Imagery (P18-1)

Copied to clipboard

Challenge: Existing models of geometric reasoning are based on visual representations of objects and objects, but they are not based in symbols or words.
Approach: They propose a new deep network architecture that specializes in answering questions that admit latent visual representations and learns to generate and reason over such representations.
Outcome: The proposed model can generate and reason over latent visual representations and is validated by two synthetic benchmarks.
From the Least to the Most: Building a Plug-and-Play Visual Reasoner via Data Synthesis (2024.emnlp-main)

Copied to clipboard

Challenge: Existing models consisting of multiple steps of visual and language processing are limited in the visual and visual processing community . a visual reasoner is a plug-and-play approach that can be used to improve VLMs' reasoning abilities.
Approach: They propose a least-to-most visual reasoning paradigm that divides a question into sub-questions and invokes external tools for resolving sub-questions.
Outcome: The proposed method can improve four VLMs on four VQA benchmarks.
VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Recent advances in Large Reasoning Models (LRMs) have enabled sophisticated visual reasoning capabilities by integrating reinforcement learning and Chain-of-Thought (CoT) supervision.
Approach: They propose a jailbreak framework that exploits visual reasoning chains to bypass safety mechanisms.
Outcome: The proposed framework achieves high attack success rates on leading closed-source MLRMs.
TURTLEAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics (2026.findings-acl)

Copied to clipboard

Challenge: Vision-language models have been explored for visual programming, but performance is unclear . most prior work focuses on visual programming for productivity .
Approach: They propose a visual programming benchmark that uses visual programming to evaluate VLMs.
Outcome: The proposed model improves on GPT-5, GPT-4o, and Qwen2-VL-72B on real-world tasks by 20% . the proposed model is based on 823 visual programming tasks in the Turtle Graphics domain .
ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration (2025.emnlp-main)

Copied to clipboard

Challenge: Multimodal Large Language Models (MLLMs) have shown impressive capabilities in vision-language understanding but their visual input remains fixed throughout the reasoning process.
Approach: They propose a model-agnostic tree search algorithm tailored for vision-level reasoning that allows MLLMs to explore textual tokens while visual input remains fixed throughout reasoning process.
Outcome: The proposed algorithm outperforms strong large models such as GPT-4o on high-resolution benchmarks and improves performance on a series of elaborate high-level benchmarks.
LEO-MINI: An Efficient Multimodal Large Language Model using Conditional Token Reduction and Mixture of Multi-Modal Experts (2025.emnlp-main)

Copied to clipboard

Challenge: Recent approaches to reduce visual tokens have been criticized for their computational efficiency and lack of visual reasoning capabilities.
Approach: They propose a novel multi-modal large language model that reduces the number of visual tokens and simultaneously boosts visual reasoning capabilities.
Outcome: The proposed model significantly reduces the number of visual tokens and boosts visual reasoning capabilities.
Beyond Embeddings: The Promise of Visual Table in Visual Reasoning (2024.emnlp-main)

Copied to clipboard

Challenge: Visual representation learning has been a cornerstone in computer vision for decades.
Approach: They propose a visual representation tailored for visual reasoning that provides instance-level world knowledge and detailed attributes that are essential for visual reason.
Outcome: The proposed visual tables outperform existing models on 11 visual reasoning benchmarks.
MDocRAG-RL: Empowering Multi-Modal Document RAG via Complex Visual Reasoning with Reinforcement Learning (2026.findings-acl)

Copied to clipboard

Challenge: Existing RAG systems produce suboptimal embeddings and naively insert images into context without adequate visual perception, limiting reasoning capabilities.
Approach: They propose a novel RAG framework for complex visual reasoning that integrates multimodal large language models with external knowledge to enhance retrieval efficiency.
Outcome: The proposed framework achieves state-of-the-art performance on multiple benchmarks.
Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models (2025.emnlp-main)

Copied to clipboard

Challenge: Recent advances in text-only "slow thinking" reasoning have prompted efforts to transfer this capability to vision-language models (VLMs).
Approach: They propose a VRM Reflection-V which enhances visual reflection based on reasoning data for cold-start and reward design for reinforcement learning.
Outcome: The proposed model improves visual reflection for cold-start and reward design for reinforcement learning (RL) it maintains a stronger and more consistent reliance on visual information during visual reasoning, indicating effective enhancement in visual reflection capabilities.
CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detection (2025.findings-acl)

Copied to clipboard

Challenge: Multimodal large language models have demonstrated impressive capabilities in visual reasoning and text generation.
Approach: They propose a multimodal large language model that captures deeper relationships between images and text . they propose CMIE, which uses a Coexistence Relationship Generation strategy and an AS mechanism to detect misinformation.
Outcome: The proposed framework outperforms existing methods in detecting out-of-context misinformation.
Mapping natural language commands to web elements (D18-1)

Copied to clipboard

Challenge: a dataset of over 50,000 natural language commands captures various phenomena, including functional references, relational reasoning, and visual reasoning.
Approach: They propose a task that requires the user to choose the correct element on a web page . they use a dataset of over 50,000 natural language commands to map these to web pages .
Outcome: The proposed task can be viewed as a reference game based on a dataset of over 50,000 natural language commands .
VDebugger: Harnessing Execution Feedback for Debugging Visual Programs (2024.findings-emnlp)

Copied to clipboard

Challenge: Visual programs are executable code generated by large language models to address visual reasoning problems.
Approach: They propose a critic-refiner framework that localizes and debugs visual programs by tracking execution step by step.
Outcome: The proposed framework detects and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy.
Can you SPLICE it together? A Human Curated Benchmark for Probing Visual Reasoning in VLMs (2025.findings-emnlp)

Copied to clipboard

Challenge: SPLICE is a benchmark designed to probe event-based reasoning across multiple dimensions.
Approach: They introduce a human-curated benchmark to probe event-based reasoning across multiple dimensions.
Outcome: The proposed benchmark includes 3,381 human-filtered videos spanning 12 categories and 180 sub-categories . results show that state-of-the-art vision-language models struggle to match human performance .
Forgotten Polygons: Multimodal Large Language Models are Shape-Blind (2025.findings-acl)

Copied to clipboard

Challenge: Multimodal Large Language Models struggle with visual reasoning, despite strong performance on vision-language tasks.
Approach: They propose a visually cued chain-of-thought prompting that enhances multi-step mathematical reasoning by explicitly referencing visual annotations in diagrams.
Outcome: The proposed model improves GPT-4o's accuracy on an irregular polygon side-counting task from 7% to 93%.
CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space (2025.emnlp-main)

Copied to clipboard

Challenge: Embodied Question Answering (EQA) tasks are primarily focused on indoor environments, leaving the complexities of urban settings unexplored.
Approach: They propose a task where an embodied agent answers open-vocabulary questions in dynamic city spaces.
Outcome: The proposed agent achieves 60.7% of human-level answering accuracy compared to baselines . the proposed agent outperforms existing agents in open-ended city spaces .
A Corpus for Reasoning about Natural Language Grounded in Photographs (P19-1)

Copied to clipboard

Challenge: a dataset for visual reasoning with natural language and images is available.
Approach: They propose a dataset for joint reasoning about natural language and images . they crowdsource 107,292 examples of English sentences paired with web photographs .
Outcome: The proposed dataset combines 107,292 examples of English sentences with web photographs . Qualitative analysis shows the data requires compositional joint reasoning .
Cross-Modality Relevance for Reasoning on Language and Vision (2020.acl-main)

Copied to clipboard

Challenge: Existing approaches to learn and reason over language and vision data for downstream tasks such as visual question answering (VQA) and natural language for visual reasoning (NLVR)
Approach: They propose a cross-modality relevance module that is used in an end-to-end framework to learn the relevance representation between components of various input modalities under supervision of a target task.
Outcome: The proposed approach shows competitive performance on two different language and vision tasks using public benchmarks and improves the state-of-the-art published results.
MM-CRITIC: A Holistic Evaluation of Large Multimodal Models as Multimodal Critique (2025.findings-emnlp)

Copied to clipboard

Challenge: e MM-CRITIC is a holistic benchmark for evaluating the critique ability of Large Multimodal Models (LMMs) covering 8 main task types and over 500 tasks, covering 4471 samples.
Approach: They introduce a holistic benchmark for evaluating the critique ability of Large Multimodal Models across multiple dimensions: basic, correction, and comparison.
Outcome: The proposed benchmark covers 8 main task types and over 500 tasks and is composed of 4471 samples.
ChartAgent: A Multimodal Agent for Visually Grounded Reasoning in Complex Chart Question Answering (2026.acl-long)

Copied to clipboard

Challenge: Recent multimodal LLMs have shown promise in chart-based visual question answering, but their performance declines sharply on unannotated charts.
Approach: They propose a novel agentic framework that explicitly performs visual reasoning directly within the chart’s spatial domain.
Outcome: The proposed framework achieves state-of-the-art accuracy on the ChartBench and ChartX benchmarks surpassing prior methods by up to 16.07% absolute gain overall and 17.31% on numerically intensive queries.
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension (2024.emnlp-main)

Copied to clipboard

Challenge: Referring Expression Comprehension (REC) is a cross-modal task that objectively evaluates the capabilities of language understanding, image comprehension, and language-to-image grounding.
Approach: They propose to use a new reference expression comprehension (REC) dataset to evaluate the capabilities of language understanding, image comprehension, and language-to-image grounding.
Outcome: The proposed model is able to reject scenarios where the target object is not visible in the image, a key aspect often overlooked in existing models and approaches.
ATLAS: Orchestrating Heterogeneous Models and Tools for Multi-Domain Complex Reasoning (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to optimize large language models with external tools are limited.
Approach: They propose a dual-path framework for dynamic tool usage in cross-domain complex reasoning . they exploit empirical priors for domain-specific alignment and RL-based multi-step routing .
Outcome: The proposed framework outperforms closed-source models and existing methods on in-distribution and out-of-distortion tasks.
Beyond the Last Frame: Process-aware Evaluation for Generative Video Reasoning (2026.acl-long)

Copied to clipboard

Challenge: Existing evaluation frameworks often rely on single-frame assessments, which can lead to outcome-hacking.
Approach: They propose a process-aware evaluation paradigm that uses a hierarchical rubric to evaluate the validity of the intermediate steps and the final result.
Outcome: The proposed model achieves POC@1.0 only about 20% and exhibits significant outcome-hacking.
MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct (2025.findings-acl)

Copied to clipboard

Challenge: a new framework for image-text instruction data evolution improves MLLM performance . lack of high-quality instruction data remains a major bottleneck in ML modeling .
Approach: They propose a multimodal instruction data evolution framework that iteratively enhances data quality through fine-grained perception, cognitive reasoning, and interaction evolution.
Outcome: The proposed approach improves MLLM performance in nine vision-language tasks while using significantly less data.
Whiteboard-of-Thought: Thinking Step-by-Step Across Modalities (2024.emnlp-main)

Copied to clipboard

Challenge: Large language models have shown promising results in arithmetic and symbolic reasoning by expressing intermediate reasoning in text as a chain of thought, yet struggle to extend this capability to answer text queries that are easily solved by visual reasoning.
Approach: They propose a method to unlock the visual reasoning capabilities of multimodal large language models by using a metaphorical ‘whiteboard’ to draw out reasoning steps as images and return these images back to the model for further processing.
Outcome: The proposed method shows that it can be used on four difficult tasks that involve visual and spatial reasoning with no demonstrations or specialized modules.
Seeing Culture: A Benchmark for Visual Reasoning and Grounding (2025.emnlp-main)

Copied to clipboard

Challenge: Multimodal vision-language models (VLMs) have made significant progress in cultural understanding tasks . but these datasets often fall short of providing cultural reasoning while underrepresenting many cultures.
Approach: They propose a Seeing Culture Benchmark that requires VLMs to reason on culturally rich images in two stages.
Outcome: The proposed approach requires VLMs to reason on culturally rich images in two stages . the Seeing Culture Benchmark identifies cultural reasoning shortcomings in multimodal models .
ComicVQA: A Benchmark for Visual Reasoning in Multimodal LLMs (2026.findings-acl)

Copied to clipboard

Challenge: ComicVQA is a visual reasoning benchmark for comics.
Approach: They propose a comics-based benchmark for evaluating MLLMs on visual reasoning.
Outcome: The proposed model achieves 62.6% accuracy on Missing Panel Prediction and 46.4% on Panel Sorting, compared to open-source models.
CheMM-R1: Enhancing Chemical Structure Recognition and Elucidation with Reasoning Multimodal Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing multimodal large language models lack domain-specific expertise to perform chemical tasks.
Approach: They propose a benchmark dataset for evaluating multi-step multimodal reasoning capacities in the chemistry domain.
Outcome: The proposed model surpasses existing models in all CheMM-Bench tasks.
Inject to Heal: Alleviating hallucination in LVLMs via Context Embedding Injection (2026.findings-acl)

Copied to clipboard

Challenge: a large vision-language model can generate hallucinations inconsistent with visual input . a lightweight method that embeds the last input token as a grounding signal reduces the likelihood of hallucinosity.
Approach: They propose a training-free mitigation strategy that harnesses the hidden state of the last input token as a grounding signal to maintain visual fidelity throughout decoding and curb hallucinations.
Outcome: The proposed method outperforms state-of-the-art methods on CHAIR, AMBER, and MMHal benchmarks.
Revealing the Seen, Imagining the Beyond: A Survey of Image-Grounded Chain-of-Thought Reasoning in Multimodal LLMs (2026.acl-long)

Copied to clipboard

Challenge: Recent advances in Multimodal Large Language Models (MLLMs) have shifted visual reasoning from tool-calling to end-to-end perceptionreasoning.
Approach: They synthesize the emerging paradigm of Image-Grounded Chain-of-Thought (IG-CoT) they propose a method-centric taxonomy covering prompting, supervised fine-tuning, and reinforcement learning .
Outcome: The proposed model is based on a method-centric taxonomy and benchmarks.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations